init_parallel
init_ray
INFO: Pandarallel will run on 8 workers.
INFO: Pandarallel will use Memory file system to transfer data between the main process and workers.
init_pandarallel
use device: cuda
init data
try train_iterate_count 1000
try validate_iterate_count 200
train try get_sample_data: 64000000
stock_per_record 17317
init_stock_data_cache 2012-06-03 00:00:00 2022-06-01 self.datas.shape (701260, 7)
datas[StockUtils.CODE]: 425
per_stock_record_count: 17386
CONTEXT.Pool(nb_workers) 8
stock_record_list.shape: (701260, 8)
CONTEXT.Pool(nb_workers) 8
normalize_by_index (701260, 14) (11382, 7)
before reshape_timelen (701260, 14)
reshape_timelen==>init 2023-07-04 19:41:56-2023-07-04 19:41:44=>12s
after reshape_timelen (672310, 771)
reshape_timelen :  2023-07-04 19:57:35-2023-07-04 19:41:56=>939s
normalize_data :  2023-07-04 20:02:57-2023-07-04 19:57:35=>321s
LocalConfig.MODE_TO_SAVE_X_INDEX validation train
get_change_data (672310, 769)
open_1_
to_xy :  2023-07-04 20:03:08-2023-07-04 20:02:57=>10s
get_stock_data_reshape_batch: 2023-07-04 20:03:08-2023-07-04 19:41:54=>1273s
ts_data: (672310, 601)
###### sample total count: (672310, 601)
###### filter count: (65554, 602)
###### crosstab:
change_class   False   True   total
filter                             
0             575329  31427  606756
1               1153     65    1218
2                287     13     300
4                 68      3      71
16               233     12     245
18                30      0      30
20                37      1      38
22                42      0      42
64               724     54     778
66                63      2      65
68                19      1      20
80                53      3      56
82                15      1      16
84                17      1      18
86                26      1      27
256              455     19     474
1024               8      0       8
4096            2041     79    2120
4097               3      0       3
4352             412     22     434
8192           50596   3002   53598
8193              19      3      22
8194            1589     61    1650
8196             414     25     439
8208             801     36     837
8210             382     23     405
8212             176      8     184
8214             327     12     339
8256             693     59     752
8258             434     17     451
8260              92      7      99
8272             182      9     191
8274             271     10     281
8276              87      9      96
8278             193      9     202
8448               2      0       2
9216               1      0       1
12288             33      4      37
12544              4      1       5
total         637311  34999  672310
change_class     False     True 
filter                          
0             0.948205  0.051795
1             0.946634  0.053366
2             0.956667  0.043333
4             0.957746  0.042254
16            0.951020  0.048980
18            1.000000  0.000000
20            0.973684  0.026316
22            1.000000  0.000000
64            0.930591  0.069409
66            0.969231  0.030769
68            0.950000  0.050000
80            0.946429  0.053571
82            0.937500  0.062500
84            0.944444  0.055556
86            0.962963  0.037037
256           0.959916  0.040084
1024          1.000000  0.000000
4096          0.962736  0.037264
4097          1.000000  0.000000
4352          0.949309  0.050691
8192          0.943990  0.056010
8193          0.863636  0.136364
8194          0.963030  0.036970
8196          0.943052  0.056948
8208          0.956989  0.043011
8210          0.943210  0.056790
8212          0.956522  0.043478
8214          0.964602  0.035398
8256          0.921543  0.078457
8258          0.962306  0.037694
8260          0.929293  0.070707
8272          0.952880  0.047120
8274          0.964413  0.035587
8276          0.906250  0.093750
8278          0.955446  0.044554
8448          1.000000  0.000000
9216          1.000000  0.000000
12288         0.891892  0.108108
12544         0.800000  0.200000
total         0.947942  0.052058
###### change_class total count: (34999, 603)
###### change_class / total : 34999 / 672310 0.05205783046511282
###### change_class & filter / filter : 3572 / 65554 0.054489428562711655
###### change_class & filter / total : 3572 / 672310 0.005313025241332123
###### augmentation count: (3572, 603)
stock_init_format_filter0: 2023-07-04 20:03:41-2023-07-04 19:41:49=>1312s
init_stock_data_cache 2012-06-03 00:00:00 2022-06-01 self.datas.shape (709756, 7)
datas[StockUtils.CODE]: 427
per_stock_record_count: 17386
CONTEXT.Pool(nb_workers) 8
stock_record_list.shape: (709756, 8)
CONTEXT.Pool(nb_workers) 8
normalize_by_index (709756, 14) (11382, 7)
before reshape_timelen (709756, 14)
reshape_timelen==>init 2023-07-04 20:03:50-2023-07-04 19:41:44=>1325s
after reshape_timelen (680596, 771)
reshape_timelen :  2023-07-04 20:21:15-2023-07-04 20:03:50=>1045s
